Papers with BBQ dataset
Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Despite efforts to mitigate social bias in large language models, representational harms such as stereotyping continue to exist in both open and closed-source models. |
| Approach: | They propose a method to modify model activations in forward passes by applying steering vectors to a BBQ dataset and comparing their results to bias mitigation methods. |
| Outcome: | The proposed method outperforms 3 other bias mitigation methods on the BBQ dataset and shows the lowest impact on MMLU scores. |
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |